{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "1d233a6c",
   "metadata": {},
   "source": [
    "## Graph Convolution LSTM\n",
    "Standard LSTM cell with Chebyshev graph convolution. <br />\n",
    "Very similar to the GConvLSTM, but seems to perform worse on the testing data. <br />\n",
    "\n",
    "Usage is analogous to a standard pytorch LSTMCell. <br />\n",
    "\n",
    "I recommend using GConvLSTM instead of this one.<br />"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "33eaec51",
   "metadata": {},
   "source": [
    "### GCLSTM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "7e473b3a",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 200/200 [22:00<00:00,  6.60s/it]\n",
      "C:\\Users\\TBROFIP4\\Miniconda3\\envs\\py37\\lib\\site-packages\\ipykernel_launcher.py:113: UserWarning: Matplotlib is currently using module://matplotlib_inline.backend_inline, which is a non-GUI backend, so cannot show the figure.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train MSE: 0.1273\n",
      "Test MSE: 0.8711\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABC90lEQVR4nO3dd3gc1dX48e/ZVe/VtizLtmTLvWEbg7ENJvQSTA8koQQSUiCUkLwhLyUJeUNII+UXkmBKAgRCS0IJpjdjsMG9N1m2ZMuSrd7bau/vjxnJK1llZWuLtOfzPPvs7szdnaOVNGdvmXvFGINSSqnQ5Qh0AEoppQJLE4FSSoU4TQRKKRXiNBEopVSI00SglFIhThOBUkqFOE0ESg0yIvKhiBgRuT7QsaihQROBChoiss8+wV0c6FiCgYgstj+PfV12vQT8Adjm/6jUUBQW6ACUUiAi4caYVm/KGmP+5Ot4VGjRGoEaNETkEhFZLSK1IlIgIg+LSJK9L0JEHhWREhFpFpH9IvKavU9E5AF7W7Nd5i0RSe3hOLEi8msR2SMidSKyQUSusffliIhbRMpFJNzeNsb+5l5uxxEmIv8jIttFpF5EtonITR7v/xO7/Esi8oKINAJf6RLDYuAD+2n7+xt7X6emIRH5u/38KRF5Q0QaReRtO65/2TGsFJFsj/efJiKvi8hhESm1y40egF+TGoQ0EahBQUTOB/4NzLDva4HvAM/ZRa4Fvg6UAY8Da4FT7H1nAD8C2ux9y4HpQHwPh/sb8H27/AtALvCUiFxtjMkHPgFSgDPt8lfa988bY1qAnwG/BAR4FogCHhGR67oc5zJgHPA0UNJl3wHgX/bjWqymoD/0EG+7rwJ1QAVwFrARSALygZPtuBCREfZncBawAvgQuBR4S0Qi+ziGGoI0EajB4hb7/gFjzHXAYsAFnCMiE4Bwe/9m4Bnga8Awe1v7vjysE/stQCZQ2PUgIjIMuMJ+epYx5gbgf+3n37Xvn7Lvv2TftyeCp0REPGL9FKgHttjPv93lcPnAScaYm4wxb3ruMMbkAe1NQBXGmNuNMbd3jbeL940xVwCP2s8bsU727fGfYN9fAyRjfR6FQBFQCkwCTu/jGGoI0j4CNViMte+3AxhjykSkDBgBjME6OS8GlgBXAQZ4V0QuAd4G/ox1AmxvblkDXAQU93CcRmNMgf14h30/xr5/AfgjcLGITALmAruMMatEJB2Is8t9rct7j+/y/HNjjKuvH7wfttv3VfZ9njHGLSK19vNY+36sfT/ZvvUWowoBWiNQg8U++34SgN2+n2ZvKwBcxpgvAQlYJ7d3sb4NXwo4sb6lJ2Gd6J7COnl/vZfjRHu0mU/0OA7GmGrgFSARWGrva68llGHVAgBmGmPEGCNY/2tzuxyruY+fuc2+9/b/tK2P5+322ff/aY/PjjEDq+lMhRhNBCoY/VJEVnncTgUetvf9r4j8HatdOwx4xxizC7haRLZjte/fhtUHANa341OAvVhNRt8DFnjs68QYcxhreCbAOyLyBPCA/dxztE77iX8RVu3jafv1xiPWt+0O7H9iNQP9pF+fAuy370eJyGMi8sN+vr4nz2D97JfYneaPiMi79vGGD9Ax1CCiTUMqGE3o8jzFGPOyiFwJ3AVcjtUh+ghWJzDATqxv4+djdQIXA/8H/BerFrAbq9M4yS73V458m+/qBqyT4iVY/QB7gIeMMc96lHkLq4N3BPCRMcazv+EeoBy4HqsDtwZYDzzv5c8PgDFmn4j8BqvmciOwFasT+rgYYw6KyGnAz4F5wEKsvoKHsT4bFWJEF6ZRSqnQpk1DSikV4jQRKKVUiNNEoJRSIU4TgVJKhbhBN2ooLS3NjB07NtBhKKXUoLJ27doyY0x6d/sGXSIYO3Ysa9asCXQYSik1qIhIQU/7tGlIKaVCnCYCpZQKcZoIlFIqxGkiUEqpEKeJQCmlQpwmAqWUCnGaCJRSKsRpIujG2oJK1hdWBjoMpZTyC00EXbja3Hz7H2t5YNn2vgsrpdQQMOiuLPaVN7cUkxAVTkNLG4drm4mN1I9GKRUa9GwHrC2o4DvPrCPc6SB3uLXueEV9S4CjUkop/wipRNDmNjgd0mlbfbOLO57fSEZiNC63my1FNcREOKlubKW1zU24U1vPlFJDW8ic5Z5auY+THniXFpe70/Y3tpRQWNHAry+fwZ+/MoeTslO4dv5YAKoaWgMQqVJK+VfIJIKMxGjK6lpYs6+i0/adJTVEhjk4KSeVOWOSef6b85mWmQBo85BSKjSETCI4ZVwq4U7ho12luN2GgvJ6AHYdqmP8sLhOTUYpsRGAJgKlVGgImT6C2MgwThybwoc7S4kKd/KnD/L46AeL2XWolpNzUjuV1USglAolIVMjAFg8MZ2dh2r5y0d7aHMb3txSQnF1ExOGx3cq15EIGjQRKKWGvhBLBMOsBwYSo8N5aqW1YM8Ee8hou+QYOxHUaSJQSg19IdM0BJA7LI65Y5JZPDGd3YfreGXDQYCjagThTgcJUWFUao1AKRUCQioRiAgvffsUAF5YvZ9XNhwkJsJJZlL0UWVTYiMo1z4CpVQICKmmIU+njLc6iHOHxeHocpEZWImgor7Z32EppZTf+TQRiMi5IrJTRPJE5K5u9v9ORDbYt10iUuXLeDyNSo5hWmYCJ45N6Xa/lQj0gjKl1NDns6YhEXECDwNnAQeA1SLyqjFmW3sZY8wdHuW/C5zgq3i685/vLMApR9cGwEoEm4uq/RmOUkoFhC9rBPOAPGNMvjGmBXgOWNJL+auBf/ownqOEOx3dNgsBpMRGUlnfijHGnyEppZTf+TIRZAL7PZ4fsLcdRUTGANnA+z3sv0lE1ojImtLS0gEPtDspseG0tLmpa3YBsKe0TpOCUmpICpbO4quAl4wxbd3tNMYsNcbMNcbMTU9P90tAKbGRAFTWt7L7UC1nPvQRb2875JdjK6WUP/kyERQBWR7PR9nbunMVfm4W6ktKbDgA5fXNrNpbgTGw7WBNgKNSSqmB58tEsBrIFZFsEYnAOtm/2rWQiEwCkoGVPoyl3zISrWsL9pTWs67AWr84v6w+kCEppZRP+CwRGGNcwC3AW8B24AVjzFYRuV9ELvIoehXwnAmyBviJw+MZnhDJe9sPsc5eyD6/tC7AUSml1MDz6ZXFxphlwLIu2+7r8vwnvozhWDkcwhmTh/OvtQdodrmJDHOwt6weYwzSw5BTpZQajIKlszgonTV5OM32imbnThtBQ0sbJTVNAY5KKaUGliaCXswfl0pMhJMwh7Bk1kgA8ku1n0ApNbRoIuhFVLiTC6ZncMr4NCZnWMtXaj+BUmqoCanZR4/Fry6f0fE4JsKpI4eUUkOOJoI+eHYMZ6fFatOQUmrI0aahfhiXHsfuQ7WBDkMppQaUJoJ+mDEqkYPVTRzWkUNKqSFEE0E/nDA6GYB1hVWBDUQppQaQJoJ+mDoygXCnsH5/ZaBDUUqpAaOJoB+iwp1MHZnI+oKqjm3GGD7YeZi/f7IXtzuoZslQSimv6Kihfpo9OplnPy+gtc264vj6v33OJ3nlABysbuJ/z58cyPCUUqrftEbQTyeMTqKp1c2O4lpe23iQT/LK+eG5k7h2/hiWLs/n5fU9zbStlFLBSWsE/TR3bDIi8NiKfHaW1JI7LI5vnZaDMbBscwmf5JVx8QndLsSmlFJBSWsE/ZSRGM33zpzAKxsOsqOklm+cmoOI4HAIOemx7NUrj5VSg4wmgmNwyxfGc+nsTHLSYjsmowPISdNEoJQafLRp6BiICA9dOQtXm5sw55Fcmp0WS3l9C9WNrSRGhwcwQqWU8p7WCI6DZxIAGJsWC8A+rRUopQYRTQQDKMdOBHvL6imra6bZ1RbgiJRSqm+aCAbQ6NQYRGB7cQ3n/G45dzy/AYDXNxWzpag6sMEppVQPNBEMoMgwJ6OSo/nHqgLK61tYtrmEe1/ews3PruOBZdsDHZ5SSnVLO4sH2NjUWPZXNDJheBwut+HpVQU4HcKagkqaWtuICncGOkSllOrEpzUCETlXRHaKSJ6I3NVDmStFZJuIbBWRZ30Zjz+09xNcO38sv7liJudNG8GDl06nxeVmbYFOVqeUCj69JgIRcYrIRSIysb9vLCJO4GHgPGAKcLWITOlSJhf4EbDAGDMVuL2/xwk2i3LTmZKRwCUnZDJ7dDJ/+eoczp+eQZhDWJFXFujwlFLqKL0mAmNMG/A4MP8Y3nsekGeMyTfGtADPAUu6lPkG8LAxptI+3uFjOE5QOXPKcJbdtojYyCOtbrGRYczKSuJTTQRKqSDkTdPQM8D1IjJVRFLab168LhPY7/H8gL3N0wRggoh8IiKrROTc7t5IRG4SkTUisqa0tNSLQwefBePT2FRUTXVDa6BDUUqpTrxJBLcCi4BNQKl9G6hv7mFALrAYuBp4VESSuhYyxiw1xsw1xsxNT08foEP717zsFIyBzTqMVCkVZLwZNbQcOJYVV4qALI/no+xtng4AnxljWoG9IrILKzGsPobjBbXJGQmAdY3Bwty0AEejlFJH9JkIjDGLj/G9VwO5IpKNlQCuAr7cpczLWDWBv4lIGlZTUf4xHi+opcRGMDwhku3FNYEORSmlOumzaUhEEkXk7yJyyL49ISKJfb3OGOMCbgHeArYDLxhjtorI/SJykV3sLaBcRLYBHwA/MMaUH/uPE9wmZySwTROBUirIeNM09EfgGo4061wPCPC1vl5ojFkGLOuy7T6Pxwb4nn0b8qZkJLBidxnNrjYiw/TCMqVUcPCms/g84FfGmCxjTBbwa+AC34Y1NE3OSMDlNqzeW8kf39tNdaOOIFJKBd6xTDFxLB3HiiMdxrf8cx1VDa1sO1jDX746GxGhsaWNNmOIi9RZP5RS/uVNjWAZ8AMRKRSRQuAHwOu+DWtoyk6LJSrcQVVDK/OyU3hzawnPfFYIwA//tYmv/e3zXl//1tYSrv/b51gtakopNTC8SQS3Y11UFm3fngbu8GFMQ5bTISwcn865U0fw3DdOZvboJJ5eWQDAmn0VbNxfTWubu8fXf5pXxoc7S6lpdPkrZKVUCOi1HcKeL+he4G/GmGv9E9LQ9ui1cwBrucuF49P40wd5FFc3crC6CYD80nomjojv9rXl9S0AFFU1khijS2EqpQaGN3MNXQyM80s0IUBEEBEApmUm4jbw73VHrrPzvM7glQ1F1DQd6VCusBNBcXWjn6JVSoUCb5qGPgTuE5GbReTS9puP4woJM0YlAfDCGmtKJofA9hIrEeyvaOC25zbw/OdHpmsqr7MSwcEqTQRKqYHjzRCV9usF/mjfC9bIIR0If5yGJ0SSFhdJQXkD6fGRpMdFsr24FoD9lQ3AkcQAnk1DTf4PVik1ZHmTCH7q8yhClIgwY1Qi7+84zOSMBNLiIlix25qq+qB9st9hJwa321DZoE1DSqmB501ncQLwX2PMB/4JKbRMy2xPBPGkx0Xy73VFlNc1U2w3/+QdrsPV5qau2UWb2xo2qk1DSqmBpJ3FATYj05q2aUpGApNGWBec7Sip5aD9rb+lzc3esvqOZqEIp6OjtqCUUgPBm6ahD7E6iyOB4vaNxph/+yqoUHLqhHTuPn8y50wdQV2zdX3A9uIaiqqaiIlw0tDSxo6SWkYkRgEwOSOeLQdraHMbnA4JZOhKqSFCO4sDLCLMwTdOzQEgKtxJerzVYVxc1cjJOal8tKuUHSU1hDutytu0zEQ2HqjmcG0TGYnRgQxdKTVEeJMI7kfnF/KbSSPi2V5cw8GqRhbmpjEuPZadJbVkJsUAViIAqzNZE4FSaiB4szDNT/wQh7JNzkjg8RV7aXMbMpOimTgigXUFlczKSgJgekciaGTOmOQARqqUGip67CwWkXUicpaIxNqL0Uyyt18iIhX+CzG0TM6I7xgdNDIpmpmjEimqamRHSS1xkWGMSbVqBjpySCk1UHobNTQLSAaisBajGWlvjwD6XKFMHZv2kUNgJYK5Y1MA+GDHYVJiI4iPCicpJpyCioZAhaiUGmL6ahrSvgE/G5ceR7hTaG0zjEyMIjk2gqhwB/UtbeTGRgCQkxZLfmldgCNVSg0Vfc01dAPwC6yEcIuI/BGrdqB8JCLM0ZEM0uIiCXc6OvoHUtsTQXoc+aX1AYxSKTWU9FUjOMfj8cUej7Wm4EPzslMIcwoO+zqBE8emsCq/gtS49kQQy0trD1Db1Ep8lE5HrZQ6Pr0lgtP9FoXq5O4LJuNqO5Jr2/sJUmIjAchJiwNgb1l9xwymSil1rHpMBMaYj/wZiDoiMsyJ59LFs0cnER8Vxrj0WICO+/xSTQRKqePnzXoEx0xEzhWRnSKSJyJ3dbP/ehEpFZEN9u3rvoxnsIqPCmfVj87gstmjABidGoND0A5jpdSA8ObK4mNiz1z6MHAWcABYLSKvGmO2dSn6vDHmFl/FMVTEelQRIsOcZKXEsKdMO4yVUsfPlzWCeUCeMSbfGNMCPAcs8eHxQoo1hFQTgVLq+PVYIxCR+3p7oTHm/j7eOxPY7/H8AHBSN+UuE5FTgV3AHcaY/V0LiMhNwE0Ao0eP7uOwoSEnPY6V+eW43aZjdJFSSh2L3pqGfuLx2GDNOtr+GKzJ6I7Xa8A/jTHNIvJN4EngC10LGWOWAksB5s6dq0NXsdYvaGp1s6OklikjE/p+gVJK9aC3RHCFfX86cBrwO6ympNuAlV68dxGQ5fF8lL2tgzGm3OPpY8CvvHhfBSwYnwbAirxSTQRKqePSYx+BMeZfxph/ARcAfzDGPGGMeQxrXQJvrjFYDeSKSLaIRABXAa96FhCRDI+nFwHb+/sDhKoRiVGMHxbHirzyvgsrpVQvvBk1FAn8WERGYSWOr+FFJ7MxxiUitwBvYS1i84QxZquI3A+sMca8CtwqIhcBLqACnb6iXxaOT+O51YU0u9qIDNN1gpRSx8abUUPfB1KB+4B77Md3evPmxphlxpgJxphxxpif29vus5MAxpgfGWOmGmNmGmNON8bsOLYfIzQtHJ9GU6ubtQWVHdv2VzTw/Rc3UtvUGsDIlFKDiTff7J8FxmLNNXQxMNYY85xPo1JeOSknBadD+CSvrGPb0uX5vLT2AE+vKghgZEqpwcTb6whOxBrNkwecLSIzfReS8lZ8VDgnZCWxYreVCBpb2nh5g9Uf/8SKvTS1tgUyPKXUINFnIhCR27GGeX4XGAFcCvzat2Epby0Yn8amomqqG1pZtrmY2iYXt5+ZS1ldCy+sOeqSDKWUOoo3NYLbgRc9nr8LzPZJNKrfFuWmYQx8uqeMZz4rIDstltvOyGVaZgL/WV/U9xsopUKeN4kgGdjo8TwGaxSQCgIzs5KIiwzjV2/tZF1hFTcszEZEWJSbzuYD1TS0uAIdolIqyHmTCD4Hvm0//j7W6KFVPotI9Uu408HJOSnsLatnckYCX55nTcFxUnYKLrdhXUFVYANUSgU9bxLBd4FGrCkmzgWKsZqLVJBYPHEYAD+9aCpOe96huWOtEUWf7dULzpRSvev1gjJ7KukJWB3EbnvzTmOMDkcJIledmMUp41LJSY/r2BYXGca0kQl8ll8RwMiUUoNBrzUC+4T/ODDXGLPNvmkSCDJhTkenJNBuXnYKG/ZX6TBSpVSvvGkaega4XkSmikhK+83Xganjt2B8Gi1tbh5dnh/oUJRSQcybRHArsAjYBJTat8O+DEoNjNMmpHPJCZn89p1dvL6puGN7m9vw14/2UN2g01AopbybdG45R9YgUIOIiPDgZdPZUVLLwx/kccEMa7LXrQerefCNHUQ4HdywMDvAUSqlAq3PRGCMWeyHOJSPRIY5WTwxnUeX53fMUnqwqgmAdYWV3IAmAqVCXZ+JQEQEay2B6UCUvdkYY7yagVQF3rSRibjchl0ldUwflUhxdSMA6wurAhuYUiooeNM09DDwLY5erlITwSAxLdNawWzrwWo7EVg1gqKqRkqqmxiRGNXby5VSQ5w3ncWXAM/aj28DPgB+5rOI1IDLSo4hPjKMLQerASiubqJ9vft1hZW9vFIpFQq8nWvoY/txMfAScJPPIlIDzuEQpoxMYOvBGgCKqxo5YXQyUeGOTovaKKVCkzdNQyV2uRKsBeYjgBpfBqUG3rTMRJ75rIA2t6G4uol52Sk4RbRGoJTyqkZwD7AHq0+gCahG5xoadKaOTKCp1c3uw7UcqmkiIzGKKSMT2FVSizE6OlipUObN8NF/eDzVJSoHqZlZSQC8uaUEl9uQkRiF20B9Sxuldc0Mi9cOY6VClTfDR9/vZrMxxpzhg3iUj+SkxZKRGMVLaw8AkJEYTXiYVSHcV9agiUCpEOZN09DiHm59EpFzRWSniOSJyF29lLtMRIyIzPXmfVX/iQgLxqdxoNK6hiAjKYrs1FgA9pXVBzI0pVSAeZMI0j1uE7CGkj7U14vsKawfBs4DpgBXi8iUbsrFYw1L/cz7sNWxWJSb1vE4IzGakUlRhDuFveWaCJQKZd4kAuNxqwF2Atd58bp5QJ4xJt8Y04LVv7Ckm3I/A36J1RGtfOiUcVYiiAxzkBwTTpjTQVZyjNYIlApx3gwfLePoSed2evG6TGC/x/MDwEmeBURkNpBljHldRH7Q0xuJyE3Y1y6MHj3ai0Or7qTHRzJpRDzNLjfWzCEwNi2WvZoIlApp/Z19tA3YB/zmeA8sIg6sJqbr+yprjFkKLAWYO3eujnU8DvddOIXa5iML2o9NjWXlnnKMMR3JQSkVWnw5+2gRkOXxfJS9rV08MA340D4BjQBeFZGLjDFrjvGYqg+njE/r9Dw7LYbG1jYO1TTrnENKhShvho8+0ctuY4y5sYd9q4FcEcnGSgBXAV/2eGE10HFWEpEPge9rEvCvsWnWyKG9ZfV9JoJmVxs7ims7rklQSg0N3jQNXc/RM496Pu42ERhjXCJyC/AW4ASeMMZsFZH7gTXGmFePJ3A1MMbaQ0jzSuuYPy6117L/WVfEXf/ezPt3ntbtGslKqcHJm0TwG6wRQPdjjTK6B+vbfp9XGRtjlgHLumy7r4eyi72IRQ2wUcnRpMVFsmZfBdecPKbXsvl2p/LHu8s0ESg1hHgzfPRa4HljzPvGmHeBF4ArjTFrjTFrfRue8jUR4aScFD7Lr+hzzqEDlQ2AlQiUUkOHN4mgEfiFiDwpIk8BDwCuPl6jBpGTs1MoqWlif0Vjr+Xar0pelV9Oa5vbH6EppfzAm0TwdaxkcA3wVaDB3qaGiJNyrL6BVXvLey23v6KBtLhI6ppdbDpQ5YfIlFL+0GciMMa8B4wBZtm3scaYD3wblvKn3GFxpMRG8Fl+RY9l6ppdVDa0cunsTES0eUipoaTXzmIREWNpEZEMrHH/w4F3/BKd8gsRYd7YFJbvLqW6sZXE6PCOfU2tbRyobKC9JWhaZiKTRiToymZKDSE91ghE5D3sE76I3Ig1+udXwJsico9/wlP+cuOibKoaWvj2P9bS4jrS/v/sZ4Wc+/uPO078o5KjmZ6ZwJaial3QRqkhoremoWnA6/bjb9n3PwM+Ar7hy6CU/504NoUHL53Bp3vKeXpVQcf2vWX1uNymY1tWcgzTMxOpbGilqKr3zmWl1ODQWyJIBMpFJBE4ASg0xvwEeBIY5ofYlJ9dNmcUucPi+GhXace24mrrZL+9uIbIMAdpcRFMy0wEYEuRLl2t1FDQWyLYh7VO8T/scm/a20cDvQ8vUYPWKeNSWb23oqN56GDVkdnBRyVHIyJMzkjA6RC2FFUHKkyl1ADqLRHcC0wELsCaivq39vargFU+jksFyCnj02hsbWN9odUncLC6kdEpMQCMSrbuo8Kd5A6LY7MmAqWGhB4TgTHmRaw1BU4CcowxeSIShjVx3M1+ik/52ck5qTgEPt1TTmNLG1UNrVx8QibxUWHkpMd2lJuWmagdxkoNEb1eR2CMKTfGrDbG1NnPXcaYjcaYQ/4JT/lbYnQ40zIT+XRPGQft/oHstBhevWUht585oaPctJEJlNe3UFKjC8upox2uaeILv/mQbQe1H2kw8ObKYhViThmXxvrCKvJLrUnmMhKjyU6L7XR9wazRyQCsL6wKRIgqyO08VEt+WT3//Lww0KEoL2giUEeZl52My214Y3MxACMTo48qMyUjgcgwh15YprpV2dAKwLLNxbh0Xqqgp4lAHWXOmBRE4K2tJQAMT4w8qkxEmIOZWUmsKaiktc3Nm1uKcbu1v0BZqhtaACivb+GTPTrIMNj1mQhEZKKIPCoi74jI+/btPX8EpwIjMTqcicPjqW9pIy0uksgwZ7fl5oxJZmtRNUuX5/Otf6zjtU0H/RypClbtNYL4qDBe2VDUR2kVaN7UCF7GWoXsDGCxx00NYfOyUwAYmdTz8pVzRltNSH94dzcAz6zS9mBlqWxoIT4yjPk5qWw6oMOMg503iSAF+B2QAaTbN72yeIg7cayVCDJ6Wcd49hirw7ilzc1pE9L5fF8FO0tq/RKfCm5VDa0kxYaTFBNOXZMuXxLsvEkETwHjgTisNYrbb2oIa68RZHTTUdwuJTaCicPjmZaZwO++NIuIMAfPflbQY3kVOqoaWkiKjiA+KpzaptZAh6P64M2axXdinfgv9NhmvHytGqSGJ0Rx74VTWDg+rddyj18/l4gwBymxEXxh4jDe2XaIn1w0FRHxU6QqGFU2tJIUE05cZBj1LW20uQ1Oh/5NBCtvTubL0RpASLpxYXafZdqnnQBYMD6VN7eWsL+ikdGpMb28Sg11VQ0tjE6JIT7KOsXUt7hIiArv41UqUPpMBMaYxX6IQw0BJ9tLXq7ML2N06ugAR6MCqarRqhG0J4LaJk0Ewcyb4aMiIleLyAMi8pB9+21fr7Nfe66I7BSRPBG5q5v93xKRzSKyQURWiMiUY/khVHAYPyyOtLgIVuq48ZDW5jZUN7aSFBNBXKR18tcO4+DmTdPQw1gL0xigvZHPYPUd9EhEnPZrzwIOAKtF5FVjzDaPYs8aY/5ql78IeAg4t18/gQoaIsJJOamsyq/AGKP9BCGqprEVYyA5Jpw4u0ZQ16wdxsHMm1FDlwDP2o9vAz7AWqmsL/OAPGNMvjGmBXgOWOJZwBjjOSNVLNoXMejNz0mlpKaJ03/zId95Zq3OThqCqhqtk75n01CN1giCmjeJIBn42H5cDLwE3OTF6zKB/R7PD9jbOhGRm0VkD9Z6yLd290YicpOIrBGRNaWlpd0VUUHizMnDmTQinviocJZtLuHj3WWBDkn5WaU9vURSTATxkXaNQBNBUPMmEZRgNSGVAI9hLVAzYHMUGWMeNsaMA34I3NNDmaXGmLnGmLnp6ekDdWjlAyMSo3jz9lN56dvzyUyK5jdv79RaQYipshNBckxER9NQrSaCoObNCf0eYA9Wn0ATUA3c7sXrioAsj+ej7G09eQ642Iv3VYNAZJiT28/MZdOBaq0VhJgqe56hpOhw4u2RQtpHENz6TATGmH8YY97EWrN4jDEmwxjznBfvvRrIFZFsEYnAWuLyVc8CIpLr8fQCYLf3oatg98WZIwlzCJ/t1VFEoaR9wrnkmAhiwp2IaNNQsPNm+OhYEVmNtW7xIhH5SETu7+t1xhgXcAvwFrAdeMEYs1VE7rdHCAHcIiJbRWQD8D3gumP9QVTwiQp3Mikjng37q47p9W9uKWHTgWN7rQqcqoYWHGLNPOpwCHGRYdpZHOS8GT76V6xOXgHcWFcaXwXc19cLjTHLgGVdtt3n8fi2/gSrBp+Zo5J4dcNB3G6Dox9TDNQ2tXLrc+s5KTuFp288yYcRqoFW1dBKYnR4x+87PjKMumZNBMHMmz6CU4A/eTzfg9Xer1SfZmUlUdvsIr+svmObMYYtRdU0tbYdVX5/RQONLW28uaWEFpeb9YVVtOmCN4NKZUMLyTERHc/josK0aSjIeVMjKAOm2Y+HYdUGdAUS5ZVZWUkAbNhfxdjUGFbklfGXD/fw2d4KFk9M5/HrTsTpEFxtbh7+YA9/fH83c0Ynd0xQVtfsYkdJDVNHJgbwp1D9UV7XQlLMkekk4qPCqdXO4qDmTY3gUayTvwDPYF0p/Igvg1JDx7j0OOIiw3ju80IW/PJ9rv/bavaU1nHFnFF8uLOUB5ZtB+A3b+/id+/uYs6YZD7fV8HK/HIuOcG67KR9XeRdh2q56E8rOFTTFLCfR/Wutc3NpgNVTBmZ0LEtLlJrBMHOm0nnfiEiB7FG9QD81xjzlG/DUkOFwyHMGJXIp3vKmTQinvuXTGPxxHQiw5zERobx+Iq9RIY5eHxFPpfPGcVvrpjJXf/axItrD3Dz6eP4dE8Za/ZVcu38sfz+3V1sOlDNqvxylsw66tpEFQQ27q+ivqWNBeOOTF8eFxXG/sqGAEal+uLVmgLGmCeBJ30cixqibvnCeBaMT+PGhdlEhR9Z//ieCyazp7SOP3+4h4SoMO46bxIAD1wynZtPH09WSgxzx6SwtqCSvMN1vLGlBLBqBio4fZJXjgjMH5fasS0hKkwvKAtyPSYCETm6J+8IY4zRhWmUV04Zl8Yp445e4CbM6eBPX57Nrf9cz6WzM0mLiwSsWkRWirWewewxyby+uZhrHv+MyDAHKTERuhxmEPtkTxlTRyaQ5NlZrE1DQa+3k7lgTQJ3EKjySzQq5CRGh/PkDfN63H/xrJHsKa0j73AdNy7MZuOBatYXVvoxQuWtxpY21hdW8rUFnRc0iosMp7G1DVebmzDngM1OowZQb4ngb8AVQBrWpHN/M8a845eolLKlxkXywCXTO54//EEer208SHldM/9eV8TVJ40mLlIrp8Fgw/4qWtsM83NSO22P75iK2tWppqCCR4/p2RhzI5ABfAdrzqA3RWSfiOh6ASpgJgyPB+Cnr23j58u28/dP9gY4ItWufTRX12VKdeK54NdrPc0YUw/kA3uBFqzaQbwf4lKqW5NGWH9+r260LmV5amUBLS53IENStrK6ZoCOvp52HVNR69XFQavHRCAid4vIbuB9YDzwXSDDGPOiv4JTqqvMpGhiI6yRR1fPG83h2mZe36zXNwaD0rpmIpwOEqI6N9W1z0CqNYLg1VuN4GdADlaNoAy4CHhGRF4VkVf8EZxSXTkcwpSRCYxJjeFnS6YyflgcT60s8Oq1L68vorA8sOPZS2ubuf+1bTS7ehuUNziV17WQGhdx1BKlulxl8Ourl02AcfbNk07+ogLmoStnAdbw04tmjuShd3ZRVtd8VJOEp6bWNu54YQNXnZjFLy6d4adIj/b2thKe+GQvZ00Z3mms/VDQ0+8gMdqqEVTWayIIVr3VCLJ7ueX4PjSlupeVEtNxncHiidaKdR/v7n0J06KqRoyBdQVVx3TMNzYX849V3tU8erPPnnxvKF4U114j6CozKRqHwL7y+m5epYJBb6OGCnq7+TNIpXoybWQiqbERfLizj0RQ2QjArsO11DT1/5vpI8vz+fMHeccUo6d9dtPUUEwEPdUIIsIcZKXEdJqBVgUXvbpDDWoOh3DqhHSW7yrtdbrqoiorERgDm/ZX9+sYxhh2H6qlpKbpuEcoDdUagTGG8rqWHpvnctJiyS/VRBCsNBGoQW/xxHQqG1rZXHTkBL+3rL7TymgHKhtwCIjAun5emVxU1Uh9SxtuA8XVjcccp9ttKKiwagQ7S2oxZuh0tdU0uWhpc5PWTdMQQE56HPvK6nHr2hJBSROBGvQW5aYjAh/tLMXtNvzk1a2c+dBHfOmRldTazUBFlY2MTIomd1gcK/LK+OWbO/h8b4VX77/7UF3H4/0Vx54Iiu0axaQR8dQ0uThU03zM7xVserqGoF12WiyNrW2U6BTiQUkTgRr0UmIjmDEqiQ93HWZlfjl//3QfC8an0exy8862Q4D1rT4zKZrZo5P5fG8Ff/lwD4+vyO/2/QrLG7jkz59QUm2dtHZ6NOMcz3TKBXaz0NlThgNDq3movK4FoNvOYoCc9FgAbR4KUpoI1JCweEI6G/ZX8bdP9hIXGcYjX51DZlI0r9lXIB+obCQzOZrL5oxiUW4as0cnsfnAkaakjfurOPf3yymva+bDXYdZX1jFG1uKAeuEnRYXSZhD2F9x7Ilgrz1q5uypIzred6joq0aQkxYHwN6yum73q8DSRKCGhNMmpmMMvLv9MOdNG0F0hJMLZ2Tw8e4ySmubOVTTxKjkGE4cm8LTN57EedMyOFjdRLl9Anty5T52lNTy6Z5ytth9DR/tskYi7TpUy+SMeEYmRbO/8tibhgrKG4gIczAlI4G0uEieX72fv360Z0hMkdFXIhieEElMhJM9WiMISj5NBCJyrojsFJE8Ebmrm/3fE5FtIrJJRN4TkTG+jEcNXTNHJZFsr5PbvsTlhTNG4nIbHv04H7eBUUnRHeWnZVprIG8uqqahxcVb9qI36wor2VJUA8DKPeU0tLjYfaiOCcPjyUqJ5sBxNA3tK6tnTEoMDodw3fwxNLvcPPjGjgG5PiHQyupaEKHjd9CViJCdFsteHUIalHyWCETECTwMnAdMAa4WkSldiq0H5hpjZgAvAb/yVTxqaHM6hDMnDycrJZqT7GmQp2UmMCsricdXWDOUZiYfSQRTM601dbcUVfPOtkPUt7SRGB3OqvwKdh2qZUpGAs0uN499vJdml5uJw+MZlRRzzJ3F1Y3WqKaxaVZb+XfPyGX5/5zOvOwUli7PH/RTTpTVNZMSE9HregM56XHka9NQUPJljWAekGeMyTfGtADPAUs8CxhjPjDGtH/FWgWM8mE8aoj72cXT+O8ti3A6rLluRIS7zpvUcX3BKI9EkBAVTnZaLBsPVPPc5/vJTIrmqhOz2F5cg8tt+PqibCLCHDz0zi5SYyNYNCGNrJRoyuqaaWzp30m7tqmVax//jLK6Zq6bP7bTvltOH09JTRP/Xld0fD98gJXVNvfYUdxuTEoMB6uaaG0b/E1hQ40vE0EmsN/j+QF7W09uBN7oboeI3CQia0RkTWlp71eQqtAVFe4ksUvTxMk5qSyemE6YQ8hIjO60b3pmIu9sO8TK/HJuWJjNnDHJHftOHJvCBdMzmDEqkVduWUBGYnTHtBY7Smr6NaXy/3s/j81F1fzlK3NYmNt5yc5FuWlMHZnA86v39/Dq7hlj+OFLm1i+Kzj+H8rre76YrN3o1Bja3IaDVcfez6J8IyiWdhKRrwJzgdO622+MWQosBZg7d65ekaL65bdXzGRnSS0RYZ2/98wYlcirGw/ytQVjuWHBWMrrrSGQidHhjEqO5qErZ3aaSXNUspUILvnzp2SnxfL+nacdNdNmVxX1LfxjVQFLZmVypj1s1JOIcMq4VJ5cWUBrm5twL5dy3FZcw/Nr9lNc08SpE9K9eo0vlVQ3cVJ2Sq9lRtuJtLCigTGpsf4IS3nJl4mgCGtls3aj7G2diMiZwN3AacaYoXOFjQoaqXGRnDL+6G+rV56YRUZiNOdNG4GIkBYXSU5aLJnJ0d2e4KdnJnLd/DGU1bXw+uZi1hVWdapFtCura+b1TcXsPlxLZX0rja1tfGdx1wl8j5iWmUiLy03e4TomZyR49TO9tdW6PmLVnnLqml0BXa7T1eampKaJkUnRvZYbY69cVlDewKJcf0SmvOXLv57VQK6IZGMlgKuAL3sWEJETgEeAc40xh30Yi1JHSYgK54IZGZ22PXLNHKLCnd2Wjwhz8NMl06hpauWdbYd4fVPxUYmg2dXGBX/8mEM1zUSHO2lsbeOC6RnkDu95YT/PEUzeJoK3t5aQGB1OdWMrK3aXsnjiMMIcEpDF4Q/XNtPmNn0mguHxUUSEOY7rWgzlGz77qzHGuIBbgLeA7cALxpitInK/iFxkF/s1EAe8KCIbRORVX8WjlDdyh8d39AX0JCEqnNMmprNsc/FRc+esLajkUE0zv/vSTLb+9BzevH0Rv7y89/UPslNjiY1wsrXIu8nwCssb2FFSy7dOG0dCVBhLl+cz9//e5ffv7vbq9QOtfUI/z1FZ3XE4hKzkaAoCvDiQOppP65PGmGXAsi7b7vN4fKYvj6+Ur1w4I4N3th1i6cf5XHViFkkx1oiZ5bvKCHMIZ00ZgcMhTBrR9zd8h0OYOjKRzUXVPPxBHnXNLn547qQey7+73WoWOn/6CHaU1PDKBuvq6U/2lPF9Jg7AT9c/7Z2/mUlRfZYdnRLTMfGeCh5B0Vms1GBz5uThTMtM4ME3dvDgGzvISonm/iXTWL6rlDljkvvdZj81M4GnVxawrrCKmAgn3z97Yscw2K62HKxmeEIkY1Jj+frCHNpLvbm1BFeb2+/NQ+01gr6ahgDGpMayel8lxpg+O9qV/+gUE0odg9jIMF67ZSGv3LyAH5wzkQing9uf28C24ppjGsUzPTMRl9sQ5hAaWtp6nYco77B1pTPA9FGJ/P6qEzh1QjpNre6OxV+MMRz200yfRZWNJMeEExPRd/LLSomhrtlFhT1Ca9OBKhpadFH7QNNEoNQxEhFmZiVx8+njeeSaOTS1WheanXYMieCUcWlMyUjgt1fOBGB9YVW35dxuw+5DdYwfFtdpe0eH84FqXG1u7nxhIyf/4j3W7PNuqu3jcbCq0avaAFgXlYE1hHR7cQ1LHv6Ehwdg5Td1fDQRKDUAxg+L52dLprFgfCpTvBz542lEYhTLblvERTNHkhob0ePiOUVVjTS2tpE7rPMopHHpcUSFO9hcVM0dL2zk3+uLCHc6+OP7vj/JHqzqe+hou/YhpBv3V/GHd3djDLxhz/OkAkf7CJQaIFeemMWVJ2b1XbAXIsIJo5NY30MiyDtszdWTO7xzjcDpEKZkJPDS2gPUNbu486wJhDkd/PLNHWzcX8XMrKTjiqsnxhiKqhqZPy7Vq/LZabHMykri58u209pmGJsaQ35pPXmHj67lKP/RGoFSQeaE0cnsKa2nuqH1qH27D1t9B+PTjz5pTs9MpK7ZxcysJL5z+niumT+GxOhwfvra1o5mq4FW0+SirtlFppc1gjCngydvmMfUkYkkx4TzyDVzAXhrq9YKAkkTgVJB5gT72/uNT67m+dWFnfbtPlRHWlwkybFHT/B2yvg0YiOc/OqyGTgdQlxkGD+/ZBrrCqu44/kNHZPvDaSDXl5D4CkxOpwXvzWf9+9czMQR8cwclcjbmggCShOBUkFm7tgUrps/hkO1Tdz7ylYaWlw89nE+Z/z2Qz7fV0FuD00o50wdwYYfn83EEUf6Dy6cMZJ7LpjMG1tKuO259d0uglPd0MrKPeUdS3P2x55Sq6nK2z6CduFOR0cyu3DGSDYeqObj3cExgV4o0kSgVJBpn8rigUum0+Jy82leOU+tLGBPaT0F5Q1H9Q946m7Suq8vyuFH503iv5uKufs/mzvtK65u5MI/fczVj67i5F+8168RPMYYli7PJzMpmskZPU+h0Zdr5o8hJy2Wu/+zxWdNWKp3mgiUClLzslOIiXDy8Id5FFY08J3F45ifk8r50zP6fnEX3zxtHNfNH8N/1hd1LCt5uKaJLz/6GVX1rfzhqlmcP30Ev35rJ29sLvbqPZdtLmHTgWruOGsCkWHdz8/kjahwJ/93yTQKKxr4+6f7+izf5ja8vbUEY3Qi4oGiiUCpIBUZ5mRRbhrrC6twOoRvLMrhnzedzMk53o3Q6eorJ4/B5Ta8vL6I8rpmvvLYZxyqaeLvN5zIklmZPHTlLGZlJfHDf206asW0/RUNPL1yX8fcSnvL6vm/17cxYXhcx9Kgx+OUcWlMz0zkPXv6jN68vL6Im55ey4q8suM+7kCrqG9hS1E1uw7VDqpEpYlAqSD2hUnDADhlXGq3HcT9MWF4PLOyknh6VQFX/HUlhRUNPH7dicwZY60jEBXu5LYzc6lpcvHxrjKMMbjdhoYWFzf8fTX3vrKVpR/ns76wkiv++inNLjcPXTmrx6kw+muhnfTqml3c+/IW/rvpYKf99fZiQK/Z2zfur+rxvdrchpV7yvu9mtyxqGlqxRhDTVMri375Phf+vxWc/bvlnPLg+6zYHXzJqjuaCJQKYqdPGkZshJPL5wzMKq5fOjGLgvIGappaeeqGeUeN/18wLo2EqDCWbS7mD+/tZsqP3+Tihz8hr7SOWVlJ/PqtnVz5yEqiwp28+K35HVc0D4RF49NwuQ3/7/3dPL2qgPte2dqxEtxznxcy6/63eX1TccfJddOB7mdr/SSvjDN++yFXP7qKqx9dRaU9nYUvFJY3cOL/vcuLaw7w8a4y6lvauPfCKfzysuk4HcIDy7Z3WzPIO1zLVUtX+jS2/tALypQKYsPio1h771k9rpHQX5eckElNYysXzRp51NKdYHVUnz11BMs2F9PicjNheDyldc3cedYErpk/li89spLstFgevHTGUcuCHq85Y5OJCnewdHk+MRFOKupbeOzjfL580uiOC9BufW49bW7D+GFxbOlm2u7S2mZufnYdyTERfO+sCfzpgzy+8thnvPbdhQNWc/H0xCd7aXa5+efqQnLS4kiMDue6+WMIczpwuQ13/2cLawsqmTu28+ptr28qYVV+Be9sO3TcFyEOBK0RKBXkBioJtL/XN08b120SaHfB9AwaWtpIiA7nma+fxOq7z+SWL+SSGB3Om7efyl++OmfAkwBYfSLzslMxBm5YkM3500fw5w/2cMnDn9Lc6ua3V8zEGMPolBi+NDeLg9VNlNYeWdTQGMO9L2+hoaWNR6+dw61n5PLry2ewrbim3xesVdS3sOlAFXvtSfy6U9PUyotr9hMfGcb6wire3FJsrY9tj9y65IRM4qPCeHJlwVGvXVNgzQH1/o7gWI9LawRKqU4WjE9jUW4a15w85rj7JfrrnKnDWVdQybXzxwCQGB3BnsN1fOu0HC6bMwqnQ0iKCSfaTo5rCyoor29h2shEnl+znze3lnDXeZMYb8/FdOGMkfz+3d385cM9HUuS9sXtNlzx10/ZU2olgX9/5xRmj+68El2Ly81Db++ivqWNx66dy01Pr6G+pa2jTwcgJiKML83N4olP9jJhWBy1zS4+31vBE9ef2DGp4Me7S2l2tXk16mp/RUOfiyYdK00ESqlOIsIcPH3jSQE59pfnjebiWZnE2us5/OLS6Z32X2yPUKprdiECP3hxE7XNR6ax/s7icdy0KKfjudMhfPPUHO7692be33GYMyYPB2DD/ipeWLOf+y+aetT6DR/uOsye0nq++4Xx/OXDPbyz7VCnRLCusJLvv7iR/NJ6Lj0hkzOnDGfB+DQ+ySs7aubZ287Mpbimid++swsRMAZ+/vp26ppdXDgjg/9uKmb13koW5qb1+rkcrmnitF9/wN0XTOHGhdnefpxe00SglAoaItKRBHoTFxnGuPQ48g7Xcc8Fk0mMDic1LoIvTBp+VNlLZmfy2Iq93PbcBp6+cR6TMxK44/kN7C2rZ8G4tKPWrX58xV4yEqO49YxcVu+r4IMdh/nhuZMoqW7i0Y/z+fun+xiREMUT18/l9IlWDeCeC6awo6SmY6W6dvFR4fzp6hO4bHYmI5Oiuec/W/jXugMA3HpGLu9sO8S72w/1mQhe3lCE28Diif2f4twbmgiUUoPSD86ZSH2zi0tn9z6iKjLMyT9uPIkrH1nJlx/9jJlZiewtqycxOpzHVuR3SgSbDlTxSV45Pzx3EuFOB4snDuPBN3bw2saD3PniRlxtbi6fM4p7LpxCQtSRfpKJI+I7Te3hSUQ6EtQ188ewpqCSYfGR5A6L4+ypI3h+9X5uOjWH93YcZkNhFbNGJ3HFnFEdfUPGGP61tohZWUmM62aywYGgncVKqUHpnKkj+kwC7UYkRvH8N0/mjMnDWJVfwZJZI/neWRNYX1jF2gJryu/yuma+/Y91DE+I5MvzRgN0fOO//fkNjEiI4sPvn86vLp/ZKQn0x7nTRpAWF8nJOamICP9zzkTcxnD1o6u49+UtvL21hHtf3sJNT6/tmG5j68Eadh6q5bIBGkLcHU0ESqmQkJEYzZ++PJvlPzidX10+g8vnjCIhKownVuzFGMNtz22grK6ZpdfM7RgVNWF4HCMTowD4/VWzGJ16fJ21kWFOXrllAfcvmQpYS3d+67RxFJQ3cMH0DDb8+Gx+ddkMPt5dyq3/XI8xhmc+KyTC6eCLM/o/tYi3tGlIKRVS2k/mkWFw9UmjeXR5Ps+vTmNFXhk//uKUTov4iAj3fXEqza62o0YOHauuazfcfPp4JmckcPqkdJwO4coTs6hubOXny7bz4Bs7eH51IV89ecxR/Q8Dyac1AhE5V0R2ikieiNzVzf5TRWSdiLhE5HJfxqKUUl1df8pYHCL87382k5EYxZdPGn1UmXOnjWDJrOOfT6knEWEOzp02otMQ0hsWZjMzK4lHlueTEhvJnWdP9NnxwYeJQEScwMPAecAU4GoRmdKlWCFwPfCsr+JQSqmeZCRGc/70DNwGvnP6+OOaRXUgOR3Cg5dOZ3hCJD9bMpXE6IG/gM+TL5uG5gF5xph8ABF5DlgCbGsvYIzZZ+87erUMpZTygzvPnsCw+EiunOu7zthjMTkjgVU/OsOri+COly+bhjKB/R7PD9jb+k1EbhKRNSKyprRUVzFSSg2cMamx3HPhlKCpDXjyRxKAQTJqyBiz1Bgz1xgzNz3dNxdUKKVUqPJlIigCPKfVG2VvU0opFUR8mQhWA7kiki0iEcBVwKs+PJ5SSqlj4LNEYIxxAbcAbwHbgReMMVtF5H4RuQhARE4UkQPAFcAjIrLVV/EopZTqnk8vKDPGLAOWddl2n8fj1VhNRkoppQJkUHQWK6WU8h1NBEopFeI0ESilVIgTY0ygY+gXESkFjl4EtG9pQNkAhzMQNK7+Cda4IHhj07j6J1jjguOLbYwxptsLsQZdIjhWIrLGGDM30HF0pXH1T7DGBcEbm8bVP8EaF/guNm0aUkqpEKeJQCmlQlwoJYKlgQ6gBxpX/wRrXBC8sWlc/ROscYGPYguZPgKllFLdC6UagVJKqW5oIlBKqRA35BNBX+sm+zGOLBH5QES2ichWEbnN3v4TESkSkQ327fwAxbdPRDbbMayxt6WIyDsistu+H5jVu72PaaLH57JBRGpE5PZAfGYi8oSIHBaRLR7buv18xPJH+29uk4jMDkBsvxaRHfbx/yMiSfb2sSLS6PHZ/dXPcfX4uxORH9mf2U4ROcfPcT3vEdM+Edlgb/fn59XTOcL3f2fGmCF7A5zAHiAHiAA2AlMCFEsGMNt+HA/swlrL+SfA94Pgs9oHpHXZ9ivgLvvxXcAvA/y7LAHGBOIzA04FZgNb+vp8gPOBNwABTgY+C0BsZwNh9uNfesQ21rNcAOLq9ndn/y9sBCKBbPv/1umvuLrs/y1wXwA+r57OET7/OxvqNYKOdZONMS1A+7rJfmeMKTbGrLMf12JNzX1MS3f60RLgSfvxk8DFgQuFM4A9xphjuar8uBljlgMVXTb39PksAZ4yllVAkohk+DM2Y8zbxpoKHmAVAZjlt4fPrCdLgOeMMc3GmL1AHtb/r1/jEmttyCuBf/ri2L3p5Rzh87+zoZ4IBmzd5IEkImOBE4DP7E232FW7J/zd/OLBAG+LyFoRucneNtwYU2w/LgGGByY0wFrYyPOfMxg+s54+n2D7u7sB65tju2wRWS8iH4nIogDE093vLlg+s0XAIWPMbo9tfv+8upwjfP53NtQTQdARkTjgX8Dtxpga4C/AOGAWUIxVLQ2EhcaY2cB5wM0icqrnTmPVRQMy1lisFe4uAl60NwXLZ9YhkJ9Pb0TkbsAFPGNvKgZGG2NOAL4HPCsiCX4MKeh+d11cTecvHH7/vLo5R3Tw1d/ZUE8EQbVusoiEY/2CnzHG/BvAGHPIGNNmjHEDj+Kj6nBfjDFF9v1h4D92HIfaq5r2/eFAxIaVnNYZYw7ZMQbFZ0bPn09Q/N2JyPXAhcBX7BMIdtNLuf14LVZb/AR/xdTL7y7gn5mIhAGXAs+3b/P359XdOQI//J0N9UQQNOsm222PjwPbjTEPeWz3bNO7BNjS9bV+iC1WROLbH2N1NG7B+qyus4tdB7zi79hsnb6lBcNnZuvp83kVuNYe1XEyUO1RtfcLETkX+B/gImNMg8f2dBFx2o9zgFwg349x9fS7exW4SkQiRSTbjutzf8VlOxPYYYw50L7Bn59XT+cI/PF35o/e8EDesHrWd2Fl8rsDGMdCrCrdJmCDfTsfeBrYbG9/FcgIQGw5WCM2NgJb2z8nIBV4D9gNvAukBCC2WKAcSPTY5vfPDCsRFQOtWG2xN/b0+WCN4njY/pvbDMwNQGx5WO3H7X9rf7XLXmb/jjcA64Av+jmuHn93wN32Z7YTOM+fcdnb/w58q0tZf35ePZ0jfP53plNMKKVUiBvqTUNKKaX6oIlAKaVCnCYCpZQKcZoIlFIqxGkiUEqpEKeJQIU8e4ZJ0+VW5YPj/MR+78sH+r2VOh5hgQ5AqSCyHmumR4CWQAailD9pjUCpI0qxLth5F3hPRK63v8H/w56LvkxEvt9eWES+Yc8RXy8in4vIQnt7hIj8QkQK7Lnsl3c5zulirRVQKiJX2K9ZYE/E1mRv9/vslyp0aSJQ6oizsZJBKZ2n0zgda7K0EuDXIjJTRL6AtZB4KdZkZKOBV0UkFWvO+Luwrki9BeuKVE9n2O+XCDxob/sfrCu8bwbuB8oG+odTqifaNKTUEZ8B99iPK4Hp9uMnjDGPiIgLeAw4DevED/BjY8w7IjIa+F+sBUK+iDVVwJeMNa98Vw8ZY5aKyLex5q4Ba/qAC7GmFFiHNXWAUn6hNQKljigzxrxr39Z6bJcu955Ml3tvtC+K4uLI/+APsWa+3I01J88asZeXVMrXtEag1BEjReQqj+fh9v3XRKQQuNV+/hHWRGB3Aj8VkXFYJ+9KrNXAXgPmAs+LyEvADGPM7X0c+0dAM1Zz0n6s5RoTgKrj/JmU6pMmAqWOOIHOi5LcYd+/B3wHGAH8wBizEcBeye1/gIeAbcAdxphyEXkQiAa+AnwB76ZTdgPftY9RjrVmbuFx/0RKeUFnH1WqB/bCLn/DOvn/JsDhKOUz2keglFIhTmsESikV4rRGoJRSIU4TgVJKhThNBEopFeI0ESilVIjTRKCUUiHu/wPKvDo91HrzjwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "from tqdm import tqdm\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import torch\n",
    "from torch import nn\n",
    "from torch_geometric_temporal.nn.recurrent import GCLSTM\n",
    "\n",
    "from torch_geometric_temporal.dataset import ChickenpoxDatasetLoader\n",
    "from torch_geometric_temporal.signal import temporal_signal_split\n",
    "\n",
    "loader = ChickenpoxDatasetLoader()\n",
    "\n",
    "lags = 10\n",
    "stride = 1\n",
    "epochs = 200\n",
    "\n",
    "dataset = loader.get_dataset(lags)\n",
    "\n",
    "train_dataset, test_dataset = temporal_signal_split(dataset, train_ratio=0.2)\n",
    "\n",
    "### MODEL DEFINITION\n",
    "class RecurrentGCN(nn.Module):\n",
    "    def __init__(self, node_features, window_length):\n",
    "        super(RecurrentGCN, self).__init__()\n",
    "        \n",
    "        self.node_features = node_features\n",
    "        self.n = window_length\n",
    "        self.hidden_state_size = 64\n",
    "        assert self.hidden_state_size % 2 == 0\n",
    "                \n",
    "        self.recurrent = GCLSTM(node_features, self.hidden_state_size, 1)\n",
    "        self.linear1 = nn.Sequential(\n",
    "            nn.Linear(self.hidden_state_size, self.hidden_state_size//2), \n",
    "            nn.Dropout(p=0.2),\n",
    "            nn.ReLU()\n",
    "        )\n",
    "        self.out = nn.Sequential(\n",
    "            nn.Linear(self.hidden_state_size//2, 1),\n",
    "            nn.Flatten(start_dim=0, end_dim=-1)\n",
    "        )\n",
    "\n",
    "    def forward(self, window, h, c):        \n",
    "        edge_index = window.edge_index\n",
    "        edge_weight = None\n",
    "    \n",
    "        H, C = [], []\n",
    "        \n",
    "        for t in range(self.n):\n",
    "            x = window.x[:,t].unsqueeze(0).T\n",
    "            h, c = self.recurrent(x, edge_index, edge_weight, h, c)\n",
    "            \n",
    "            H.append(h.detach())\n",
    "            C.append(c.detach())\n",
    "            \n",
    "        x = self.linear1(h)\n",
    "        pred = self.out(x)\n",
    "\n",
    "        return pred, H, C\n",
    "    \n",
    "model = RecurrentGCN(node_features=1, window_length=10)\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr=0.01)\n",
    "\n",
    "\n",
    "### TRAIN\n",
    "model.train()\n",
    "\n",
    "loss_history = []\n",
    "for _ in tqdm(range(epochs)):\n",
    "    h, c = None, None\n",
    "    total_loss = 0\n",
    "    for i, window in enumerate(train_dataset):\n",
    "        optimizer.zero_grad()\n",
    "        \n",
    "        y_pred, H, C = model(window, h, c)\n",
    "\n",
    "        h = H[stride-1]\n",
    "        c = C[stride-1]\n",
    "        \n",
    "        assert y_pred.shape == window.y.shape\n",
    "        loss = torch.mean((y_pred - window.y)**2)\n",
    "        total_loss += loss.item()\n",
    "        \n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "    total_loss /= i+1\n",
    "    loss_history.append(total_loss)\n",
    "\n",
    "### TEST \n",
    "model.eval()\n",
    "loss = 0\n",
    "with torch.no_grad():\n",
    "    h, c = None, None\n",
    "    for i, window in enumerate(test_dataset):\n",
    "        y_pred, H, C = model(window, h, c)\n",
    "        h = H[stride-1]\n",
    "        c = C[stride-1]\n",
    "        \n",
    "        assert y_pred.shape == window.y.shape\n",
    "        loss += torch.mean((y_pred - window.y)**2)\n",
    "    loss /= i+1\n",
    "\n",
    "### RESULTS PLOT\n",
    "fig, ax = plt.subplots()\n",
    "\n",
    "x_ticks = np.arange(1, epochs+1)\n",
    "ax.plot(x_ticks, loss_history)\n",
    "\n",
    "# figure labels\n",
    "ax.set_title('Loss over time', fontweight='bold')\n",
    "ax.set_xlabel('Epochs', fontweight='bold')\n",
    "ax.set_ylabel('Mean Squared Error', fontweight='bold')\n",
    "plt.show()\n",
    "    \n",
    "print(\"Train MSE: {:.4f}\".format(total_loss))\n",
    "print(\"Test MSE: {:.4f}\".format(loss))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4d234a4c",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
